1. Field of the Invention
An aspect of the present invention relates to at least one of a projection system, an image processing device, and a projection method.
2. Description of the Related Art
Conventionally, attention has been attracted by a multi-projection technique for arranging projection images from a plurality of projectors so as to have mutually overlapping areas and thereby projecting one image onto a screen at a high resolution.
For example, the above-mentioned multi-projection technique is known by Japanese Patent No. 3,908,255. An image projection system in Japanese Patent No. 3,908,255 projects, from each projector onto a screen, a reference image that has four or more characteristic points with preliminarily known position coordinates. Herein, a reference image is a known image such as a grid pattern wherein, for example, bright spots or crosses are arranged at a constant interval. Then, positions of the above-mentioned characteristic points in a reference image that is imaged by a digital camera are detected, then a projection image for each projector is deformed based on such detected positions of four or more characteristic points for each projector, and an overlapping area is detected to execute a blending process. Such deformed and blending-processed images from a plurality of projectors are projected onto and arranged on a screen so as to have a mutually overlapping area, and thereby, one image is projected at a high resolution.
In a case where the above-mentioned multi-projection is executed, it is necessary to project and image reference images from respective projectors sequentially or simultaneously for a position adjustment and a scale adjustment of projection images. However, in a method that sequentially projects and images more than one reference image from respective projectors according to a conventional technique, a camera has to be fixed on a tripod mount or the like and it is necessary to execute imaging in such a manner that projection ranges of all the projectors are inside an angle of view. Therefore, it is not sufficient because a mechanism such an a tripod mount is needed and convenience is lowered. Furthermore, as the number of projectors are increased, it may be difficult to image in projection ranges of all the projectors inside an angle of view at once. For example, such a case is multi-projection on a wall of a hallway or the like.
On the other hand, in a method that projects and images reference images from projectors simultaneously according to a conventional technique, structured patterns such as bright spots or crosses in reference images that are simultaneously projected from projectors are overlapped. Hence, assignment of these patterns has to be determined by image processing. Herein, in a case where patterns from projectors are combined, it is difficult to separate, or determine assignment of, such patterns.
Furthermore, Japanese Patent No. 3497805 discloses a technique for executing calibration based on divisional imaging without providing projection ranges of all the projectors inside an angle of view as mentioned above. However, it is necessary to control a position and an orientation of a camera accurately at a time of divisional imaging in order to integrate images divided by divisional imaging in Japanese Patent No. 3497805, so that a dedicated position control device is necessary for such a camera control. Therefore, a conventional technique based on divisional imaging in Japanese Patent No. 3497805 is not sufficient from the viewpoint of ease or cost of calibration. Furthermore, Japanese Patent No. 3497805 as mentioned above is silent with respect to a problem of overlapping of structured patterns.
Furthermore, in a case where divisional imaging is executed while an imaging direction or an imaging position is shifted, it is necessary to have corrected a lens distortion of a camera in addition to accurate controlling of such a position. For example, an object to be imaged that is common to images that have been imaged while an imaging position is shifted, is imaged at different positions of a lens, so that distortions are different in respective images. Hence, it is not sufficient from the viewpoint of an increased work load for correction of a lens distortion or an increased constraint on a camera capable of being used. Furthermore, a high flatness of a projection screen is also required. If there is an irregularity on a screen, a position of an object to be imaged that is common to images that have been imaged while an imaging direction or an imaging position is shifted, is shifted based on parallax. Hence, there is a problem of an increased constraint on a flatness of a screen or a degraded image quality. Japanese Patent No. 3497805 as mentioned above is silent with respect to such a problem in correction of a lens distortion or a flatness of a screen.
A technique for simultaneously projecting and imaging, and separating in a post-process, overlapping structured patterns in stack projection for overlapping and projecting onto a projection body images from a plurality of projectors is known from Japanese Patent Application Publication No. 2012-047849. A conventional technique in Japanese Patent Application Publication No. 2012-047849 discloses a method that projects patterns at respectively different wavelength regions such as R, G, and B from projectors or projecting patterns with different light polarization characteristics therefrom, and subsequently separating superimposed patterns based on a wavelength or a light polarization characteristic. However, in a method that projects patterns at different wavelength regions, wavelength regions of R, G, and B of a camera are not usually identical to wavelength regions of R, G, and B of a projector, so that it is not easy to separate independent color signals by using a normal camera. In a method that projects patterns with different light polarization characteristics, a dedicated imaging device is necessary and there is a problem of an increased cost.
In addition, Japanese Patent Application Publication No. 2011-182076 discloses a method that simultaneously projects and images a plurality of kinds of patterns with mutually shifted phases from a plurality of projectors while a pattern arrangement is designed so as not to overlap mutual patterns. However, it is necessary to project a pattern with a certain size in order to ensure precision of pattern extraction. On the other hand, it is necessary to decrease a pattern distance in a case where a spatial density of patterns is increased to improve precision of position adjustment. Furthermore, projection is made at a distance that is extremely close to a screen in an ultra-short focus projector that has appeared in recent years, so that a projection image is readily distorted non-linearly due to focus, a sensitive installation condition, or a fine irregularity of a screen. Because of these reasons, there is a limitation to a method for simultaneous projection and imaging by a plurality of projectors while mutual overlapping of patterns is avoided.